This application claims the benefit of a Japanese Patent Application No. 2000-205280 filed Jul. 6, 2000, in the Japanese Patent Office, the disclosure of which is hereby incorporated by reference.
1. Field of the Invention
The present invention generally relates to image processing methods and apparatuses and storage media, and more particularly to an image processing apparatus and an image processing method which create a background image from a dynamic image and combine the background image with another image, and to a computer-readable storage medium which stores a program for causing a computer to combine a background image and another image in such a manner.
Recently, due to developments made in information processing techniques related to dynamic images, much attention is drawn to a technique which treats an object within a dynamic image and a background as independent object images. The ISO is working to standardize MPEG-4 as an international standard for dynamic image compression coding, and techniques related to object images, such as combining independent dynamic object images to generate one scene, are being studied for the MPEG-4.
While such studies are being made, various research have been made with respect to techniques for forming an object image by extracting only a specified object from within a dynamic image. In addition, with respect to a background of the image, various research have also been made to form a background object image by extracting only the background image from the image by removing the object within the image. A new dynamic image can be created by combining such a background object image and another dynamic object image.
2. Description of the Related Art
As methods of creating a background object image from a dynamic image in which the background is still (static background), there is a known method which obtains one background image by overlapping regions of each of the frames and having no object. According to this known method, the background regions of each of the frames can be specified and collected while viewing the dynamic image if the background is still, so as to obtain a background object image which is formed by a single static image.
In addition, a method for creating a background object image from a dynamic image in which the background moves (dynamic background) is proposed in a Japanese Laid-Open Patent Application No. 9-130803, for example. According to this proposed method, motion vectors of the background regions are used to estimate the background at portions where the background does not exist, so as to obtain a background image.
On the other hand, a Japanese Laid-Open Patent Application No. 6-338136 proposes a method for detecting motion of the background regions, so as to obtain a background image having no large distortions, even when the motion of the background differs for each of the background regions.
Furthermore, a Japanese Laid-Open Patent Application No. 8-305830 proposes a method for treating a background image as a portion of an original image when combining the background image and an object image, and combining the background image and the object image so that the object image conceals a background region where the background image does not exist. According to this proposed method, the original dynamic image is described by a structural data, and the background image and the object image are combined in a state where no hole exists in the background region, based on the structural data corresponding to the dynamic image and information related to the image which is to be obtained by combining the dynamic image and the background image.
However, the conventional image processing methods described above have the following problems.
According to the method for creating the background object image, it is a precondition that the entire background region is visible in order to be able to create the background object image. In other words, in a case where it is impossible to create a complete background such that the object image within the dynamic image constantly conceals a certain region of the background, it is only possible to obtain an incomplete background object image because the conditions for the background object image are not satisfied. With respect to such an incomplete background object image, the user must manually draw the background so as to fill in the hole portion where the background does not exist or, pixels at the hole portion must be estimated from neighboring pixels in the vicinity of the hole portion, so as to create a complete background object image in which no hole portion is visible. However, it requires a troublesome and time consuming operation to manually draw the background corresponding to the hole portion, thereby putting a considerably large load on the user. In addition, when the user manually draws the background corresponding to the hole portion, the originality of the image is lost because the manually drawn background is created artificially. On the other hand, it requires a complex and time consuming computing process to estimate the pixels at the hole portion, thereby putting a considerably large load on a processor within the image processing apparatus.
For example, it is possible to employ the method proposed in the Japanese Laid-Open Patent Application No. 8-305830 in order to fill the hole portion in the background. However, this proposed method regards the background as a portion of the original image, and it is impossible to separate only the background from the image. Accordingly, even in a case where only the background is required as a part, the entire original image including the background is obtained as one part, and the image processing efficiency becomes poor.
Therefore, according to the conventional image processing methods, there were problems in that it is impossible to generate a background object image from within a dynamic image and to carry out an image processing using the background object image as an independent image part, even with respect to a dynamic image which includes a region where the background is partially not visible.
Accordingly, it is a general object of the present invention to provide a novel and useful image processing method and apparatus and storage medium, in which the problems described above are eliminated.
Another and more specific object of the present invention is to provide an image processing method, an image processing apparatus and a computer-readable storage medium, which can generate a background object image from within a dynamic image and carry out an image processing using the background object image as an independent image part, even with respect to a dynamic image which includes a region where the background is partially not visible.
Still another object of the present invention is to provide an image processing apparatus comprising a frame specifying processor which specifies, from within a dynamic image, a frame having a background which is to be used, a background region specifying section which specifies a background region from an image of the specified frame, a background generator which generates an entire background from the specified background region and outputs a background data, and a non-background region setting section which creates a non-background region data in which a non-background region flag is set within a region where no background exists in the entire background which is generated, where the background data and the non-background region data form a background object image. According to the image processing apparatus of the present invention, it is possible to generate a background object image from within a dynamic image and carry out an image processing using the background object image as an independent image part, even with respect to a dynamic image which includes a region where the background is partially not visible.
A further object of the present invention is to provide an image processing apparatus comprising an image specifying processor which specifies frames having a background which is to be used, a background region specifying section which specifies a background region from images of the specified frames, a background generator which generates an entire background from the specified background region and outputs a background data, and a non-background region setting section which creates a non-background region data in which a non-background region flag is set within a region where no background exists in the entire background which is generated, where the background data and the non-background region data form a background object image. According to the image processing apparatus of the present invention, it is possible to generate a background object image from within a dynamic image and carry out an image processing using the background object image as an independent image part, even with respect to a dynamic image which includes a region where the background is partially not visible.
Another object of the present invention is to provide an image processing apparatus comprising a non-background region recognizing section which recognizes a non-background region within a background and obtains non-background region information, based on a non-background region data of a background object image, the non-background region data having a non-background region flag which is set within a region where no background exists in the background, an automatic combining position determination section which calculates a position of an object capable of concealing a hole in the background and determines position information, based on the non-background region information recognized by the non-background region recognizing section and position information within a dynamic image data of a dynamic image object image, and an image combining section which combines the background data of the background object image and the dynamic image data of the dynamic image object image into a single combined image, depending on the position information determined by the automatic combining position determination section. According to the image processing apparatus of the present invention, it is possible to automatically arrange an object so as to conceal a region where no background exists when combining a background object image and a dynamic image object image, thereby enabling a background having no hole to be generated in a simple manner.
Still another object of the present invention is to provide a computer-readable storage medium which stores a program for causing a computer to create a background object image, where the program causes the computer to perform the procedures of specifying, from within a dynamic image, a frame having a background which is to be used, specifying a background region from an image of the specified frame, generating an entire background from the specified background region and outputting a background data, and creating a non-background region data in which a non-background region flag is set within a region where no background exists in the entire background which is generated, where the background data and the non-background region data form the background object image. According to the computer-readable storage medium of the present invention, it is possible to generate a background object image from within a dynamic image and carry out an image processing using the background object image as an independent image part, even with respect to a dynamic image which includes a region where the background is partially not visible.
A further object of the present invention is to provide a computer-readable storage medium which stores a program for causing a computer to create a background object image, where the program causes the computer to perform the procedures of specifying frames having a background which is to be used, specifying a background region from images of the specified frames, generating an entire background from the specified background region and outputting a background data, and creating a non-background region data in which a non-background region flag is set within a region where no background exists in the entire background which is generated, where the background data and the non-background region data form the background object image. According to the computer-readable storage medium of the present invention, it is possible to generate a background object image from within a dynamic image and carry out an image processing using the background object image as an independent image part, even with respect to a dynamic image which includes a region where the background is partially not visible.
Another object of the present invention is to provide a computer-readable storage medium which stores a program for causing a computer to create a background object image, where the program causes the computer to perform the procedures of recognizing a non-background region within a background and obtaining non-background region information, based on a non-background region data of a background object image, where the non-background region data has a non-background region flag which is set within a region where no background exists in the background, calculating a position of an object capable of concealing a hole in the background and determining position information, based on the non-background region information recognized by the non-background region recognizing section and position information within a dynamic image data of a dynamic image object image, and combining the background data of the background object image and the dynamic image data of the dynamic image object image into a single combined image, depending on the position information determined by the automatic combining position determination section. According to the computer-readable storage medium of the present invention, it is possible to automatically arrange an object so as to conceal a region where no background exists when combining a background object image and a dynamic image object image, thereby enabling a background having no hole to be generated in a simple manner.
Still another object of the present invention is to provide an image processing method for creating a background object image, comprising the steps of specifying, from within a dynamic image, a frame having a background which is to be used; specifying a background region from an image of the specified frame; generating an entire background from the specified background region and outputting a background data; and creating a non-background region data in which a non-background region flag is set within a region where no background exists in the entire background which is generated, where the background data and the non-background region data form the background object image. According to the image processing method of the present invention, it is possible to generate a background object image from within a dynamic image and carry out an image processing using the background object image as an independent image part, even with respect to a dynamic image which includes a region where the background is partially not visible.
A further object of the present invention is to provide an image processing method for creating a background object image, comprising the steps of specifying frames having a background which is to be used; specifying a background region from images of the specified frames; generating an entire background from the specified background region and outputting a background data; and creating a non-background region data in which a non-background region flag is set within a region where no background exists in the entire background which is generated, where the background data and the non-background region data form the background object image. According to the image processing method of the present invention, it is possible to generate a background object image from within a dynamic image and carry out an image processing using the background object image as an independent image part, even with respect to a dynamic image which includes a region where the background is partially not visible.
Another object of the present invention is to provide an image processing method for creating a background object image, comprising the steps of recognizing a non-background region within a background and obtaining non-background region information, based on a non-background region data of a background object image, the non-background region data having a non-background region flag which is set within a region where no background exists in the background; calculating a position of an object capable of concealing a hole in the background and determining position information, based on the non-background region information recognized by the non-background region recognizing section and position information within a dynamic image data of a dynamic image object image; and combining the background data of the background object image and the dynamic image data of the dynamic image object image into a single combined image, depending on the position information determined by the automatic combining position determination section. According to the image processing method of the present invention, it is possible to automatically arrange an object so as to conceal a region where no background exists when combining a background object image and a dynamic image object image, thereby enabling a background having no hole to be generated in a simple manner.
Other objects and further features of the present invention will be apparent from the following detailed description when read in conjunction with the accompanying drawings.